1. Field of the Invention
The present invention relates to image processing apparatuses, a control method therefor, and a storage medium storing a control program therefor, and in particular, relates to an image processing apparatus that is capable of generating a video image by compositing a plurality of frames of images with high dynamic range (HDR) composition.
2. Description of the Related Art
There is an image pickup apparatus, such as a digital camera or a digital video camera, as one of image processing apparatuses. Some of image pickup apparatuses perform a composition process that obtains a high dynamic range (HDR) image by compositing a plurality of frames of images that are taken under mutually different exposure amounts (hereinafter referred to as an HDR composition process). In the HDR composition process, an image that does not include what is called halation and black collapse is obtained by compositing images with a proper exposure.
Incidentally, in the HDR composition process, a HDR image is obtained also in a video image by repeating a process that composites a plurality of taken frames into one frame according to the time series. However, a frame rate of a video image obtained by the HDR composition process becomes lower than a frame rate of an image pickup. Accordingly, in particular, when shutter speed is fast, a motion of a moving subject may not look smooth.
Furthermore, in the case of a video image, a moving subject is generally replaced with pixels in one of a plurality of frames according to the characteristics in the HDR composition process. Accordingly, when accuracy of moving-subject detection is insufficient, a user may feel strange about the composite image, because portions of the moving subject are replaced with pixels of the different frames.
In order to solve such a problem, in an HDR composition process for a video image, there is a known technique that takes two images of which exposures are different by means of long-time exposure and short-time exposure, generates an estimated image by adding a blur effect to the long-time exposed image, generates a differential image by computing a difference between the long-time exposed image and the estimated image, and encodes the short-time exposed image and the differential image, for example (see WO2010/116731). This increases encoding efficiency rather than a case where a short-time exposed image and a long-time exposed image are encoded.
Moreover, there is a technique that generates an intermediate image based on information about motions in preceding and succeeding frames in order to compensate image information in no-exposure period during one frame period of video image (see Japanese Laid-Open Patent Publication (Kokai) No. 2009-232382 (JP 2009-232382A)). In this publication, when one image is generated by compositing an original image and an intermediate image, composition weighting coefficients of the original image and the intermediate image are computed according to shutter speed.
As mentioned above, in WO2010/116731, although a difference between two frames is compensated to try to increase encoding efficiency (i.e., although the difference between frames is reduced to try to increase the encoding efficiency), it is difficult to display a video image smoothly in the HDR composition process.
Moreover, in Japanese Laid-Open Patent Publication (Kokai) No. 2009-232382 (JP 2009-232382A), since composition weighting coefficients of the original image and the intermediate image are computed according to shutter speed when one image is generated by compositing the original image and the intermediate image, the smoothness of the appearance of the video image is improved. However, the HDR effect owing to the HDR composition process is reduced.
That is, the methods described in the above-mentioned two publications are difficult to obtain a sufficient HDR effect while keeping smoothness of an appearance of a video image.